Risk-Driven Security Metrics in Agile Software Development - An Industrial Pilot Study

The need for effective and efficient information security solutions is steadily increasing in the software industry. Software and system developers require practical and systematic approaches to obtain sufficient and credible evidence of the security level in the system under development in order to guide their efforts and ensure the efficient use of resources. We present experiences of developing and using hierarchical security metrics and measurements in an industrial pilot study at Ericsson Finland. The pilot focused on risk-driven security design and implementation in the context of an Agile software development process. The pilot target was a well-established telecommunications product of Ericsson and a core component in modern mobile networks. The results of the study demonstrate the practical potential of risk-driven security metrics, particularly in offering some early visibility of security effectiveness and efficiency. Hierarchical metrics models enable the linking of security objectives with detailed measurements. Security metrics visualization was found to play a crucial role in increasing the manageability of metrics. We also found that the practical means of managing larger collections of metrics and measurements are more essential than individual security metrics. A major challenge in the use of risk-driven security metrics is the lack of evidence for security effectiveness evidence in the early phases of product development and Risk Analysis, when the needs for it are at their greatest.